Incremental Parameter Estimation of Stochastic Context-Free Grammars without Retention of the Sentence Corpus

Stochastic context free grammars (SCFGs) are often used to represent the syntax of natural languages. Most algorithms for learning these grammars from data require a corpus of sentences to be stored and repearedly processed. We are interested in how embedded agents might learn the syntax of natural languages from exposure to utterances over long periods of time. In this context, the memory and computational requirements of existing algorithms for learning SCFGs from data are prohibitive. We present an online algorithm for learning the parameters of SCFGs that computes summary statistics from sentences as they are observed. The algorithm thus requires a xed amount of space regardless of the number of sentences it processes. Despite the fact that it uses much less information than the Inside-Outside algorithm, our algorithm performs almost as well.